Method and apparatus for performing knowledge level conversion of hierarchical charts created as a learning aid

ABSTRACT

A method for converting a document which contains words and images which is intended to convey information in a traditional, linear format into a document or display in which the words and images are rearranged so that the result is laid out in a pictorial representation which makes it much easier for a person using the document to understand its content. To create the easier to understand pictorial/graphical representation, a document, in its original format, is stored so that it can be accessed by a user using a personal computer or work station. A user accesses the document in its original format, selects portions of the document, and each selected portion is assigned a category for subsequent processing. The categories which may be assigned are selected from among the following: i) structural, ii) functional, iii) procedural, iv) time based, and v) event based. Additional categories may be added as a function of the desired end result and specific sub-parts or diagrams. Once the categories have been assigned to selected portions of the document, a new document is created which displays the information in the original document in a much easier to understand pictorial/graphical representation.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention is a methodology for enabling the visualization,acquisition, understanding, utilization, creation and sharing ofinformation using document structures and visual or pictorialrepresentation of documents.

2. Description of the Prior Art

Numerous teachings exist for displaying document information in a visualor pictorial manner. See, for example, U.S. Pat. No. 4,613,946 Methodand Apparatus for Generating Hierarchical Displays, U.S. Pat. No.4,764,867 Display System and Method for Constructing and Editing aHierarchical Arrangement of Information and U.S. Pat. No. 5,644,740Method and Apparatus for Displaying Items of Information Organized in aHierarchical Structure. However, the prior art does not provide amechanism for quickly and easily assigning categories to selectedportions of document information and using the assigned categories toproduce a pictorial representation of the information appropriate to theassigned category in an intuitive and easy to understand manner.

SUMMARY OF THE INVENTION

A method and apparatus are described for converting a document whichcontains words and images which is intended to convey information in atraditional, linear format into a document or display in which the wordsand images are rearranged so that the result is laid out in a pictorialrepresentation which makes it much easier for a person using thedocument to understand its content. To create the easier to understandpictorial/graphical representation, a document, in its original format,is stored so that it can be accessed by a user using a personal computeror work station including a monitor, storage, keyboard and pointingdevice. The document in its original format may be created by a wordprocessor or the like. Using an interface which forms part of thepresent invention, a user having a computer or workstation accesses thedocument in its original format from the storage such that it isdisplayed on the monitor. The user scrolls through the document andselects portions of the document, and each selected portion is assigneda category for subsequent processing. The categories which may beassigned are selected from among the following: i) structural, ii)functional, iii) procedural, iv) time based, and v) event based.Additional categories may be added as a function of the desired endresult and specific sub-parts or diagrams. The user selects which of thefive categories to assign based on the subject matter of the text andimages. For example, if the subject matter is a description of the partsof an automobile engine and how the parts connect to each other, thestructural category would be assigned. If the subject matter is adescription of how the engine operates, the functional category would beassigned. If the subject matter is a description of how to build theengine, the procedural category would be assigned. If the subject matteris a description of the history of facts leading to the development ofthe engine, the time based and/or event based category would beassigned, depending on what one wishes to communicate or emphasize. Ifspecific dates are more important and is to be communicated, then thetime based category would be assigned. If the specific events whichoccurred are more important than the time when they occurred, then theevent based category would be assigned.

The invention provides an interface which makes the selection andassigning of categories process easy, intuitive and efficient. Once thecategories have been assigned to selected portions of the document, anew document is created which displays the information in the originaldocument in a much easier to understand pictorial/graphicalrepresentation. Specifically, portions of the text and images of theoriginal document are selected, using for example, a mouse or otherpointing device. The selected portions are then assigned a categoryselected from S T E P F, i.e., structural, time based, event based,procedural or functional. After all the desired text and images in thedocument has been assigned one of the S T E P or F categories, theselected portions are then sent to a corresponding software module forprocessing based on the assigned category. Since there are fivecategories, there are five modules. If additional categories are neededor desired, appropriate additional modules may be added. Selected imagesin the original document are sent to a sixth module for processing.After the processing of the text has been performed by one or more ofthe first five modules, and the processing of images, if any, has beenperformed by the sixth module, a seventh module is used to link theconverted text and images together. If images are not present ordesired, then no further processing needs to be performed by the sixthand/or seventh modules. Further processing, which is optional, may thenbe performed to create multiple hierarchies by an eighth module. A ninthmodule, which is also optional, may be used for editing layout, contentand/or format. A tenth module, which is also optional, may be used toperform what is referred to knowledge level conversion.

BRIEF DESCRIPTION OF THE DRAWINGS

The file of this patent contains at least one drawing executed in color.Copies of this patent with color drawing(s) will be provided by thePatent and Trademark Office upon request and payment of the necessaryfee.

FIG. 1 shows the icons representing processing steps performed bypre-conversion module I.

FIG. 2 shows the icons representing processing steps performed bypre-conversion module II.

FIG. 3 shows the icons representing processing steps performed by module1 which converts information assigned to the structural category.

FIG. 3a shows a representative chart formed by the processing of module1, steps 1-6.

FIG. 3b shows a representative chart formed by the processing of module1, steps 1-9.

FIG. 4 shows the icons representing processing steps performed by module2 which converts information assigned to the functional category.

FIG. 4a shows a representative chart formed by the processing of module2.

FIG. 5a shows the icons representing processing steps performed bymodule 3 a which converts information assigned to the proceduralcategory.

FIGS. 5b-5 g show the results of the various processing steps performedby module 3 a.

FIG. 5h shows the icons representing processing steps performed bymodule 3 b which converts information assigned to a special case of theprocedural category.

FIGS. 5i-5 k show the results of the various processing steps performedby module 3 b which converts information assigned to a special case ofthe procedural category.

FIG. 6 show the icons representing processing steps performed by module4 which converts information assigned to the time based category.

FIGS. 6a-6 d show the results of the various processing steps performedby module 4.

FIG. 7 show the icons representing processing steps performed by module5 which converts information assigned to the event based category.

FIGS. 7a-7 e show the results of the various processing steps performedby module 5.

FIG. 8a shows the icons representing the processing steps performed bymodule 6 which converts images in a first format.

FIGS. 8b-8 d show the results of the various processing steps performedby module 6 for converting images in the first format.

FIG. 8e shows the icons representing the processing steps performed bymodule 6 which converts images in a second format.

FIGS. 8f-8 j show the results of the various processing steps performedby module 6 for converting images in the second format.

FIG. 9 shows the icons representing the processing steps performed bymodule 7 which links text converted by one of modules 1-5 and imagesconverted by module 6.

FIGS. 9a-9 g show the results of the various processing steps performedby module 7.

FIG. 10 shows the icons representing the processing steps performed bymodule 8 which creates multiple hierarchies useful for collaboration.

FIG. 10a-10 g show the results of the various processing steps performedby module 8.

FIG. 11 shows the icons representing the processing steps performed bymodule 9 used for editing layout, content and/or format.

FIG. 12 shows the icons representing the processing steps performed bymodule 10.

FIGS. 12a-12 e show the results of the various processing stepsperformed by module 10.

DETAILED DESCRIPTION OF THE INVENTION

A method and system are disclosed for converting a document whichcontains words and images which is intended to convey information in atraditional, linear format into a document or display in which the wordsand images are rearranged so that the result is laid out in form whichmakes it much easier for a person using the document to understand itscontent. The invention is implemented as a software system whichincludes a set of modules. The software system may be implemented to runon commercially available personal computer systems provided by avariety of vendors. Such personal computer systems typically include aprocessor, memory, including mass storage devices, keyboard and mouse orother pointing devices and a monitor or other display. In the preferredembodiment, there is a module for processing each one of five possibleassigned categories. The specifics of the five possible categories isexplained below, although there may be additional categories which maybe assigned. There is a pre-conversion module I which is used to selecttext and images and assign one of the five categories to the selectedportions. A pre-conversion module II is then used to separate the textfrom images in each assigned category. The text is then sent to one of aset of five modules corresponding to the five possible assignedcategories, i.e., structural (S), time based (T), event based (E),procedural (P) and functional (F). Processing of the selected images, ifany, is performed by a sixth module. Additional modules may then be usedto link the text and images and to perform other optional processing asdescribed in detail below.

Pre-conversion Module I:

Pre-conversion module I operates to:

1) select “units” of information to be conveyed and

2) save the “units” in one of five module databases (S, T, E, P, or F)for further processing.

A unit is defined to be an amount of information that covers a single,high level concept, but is not at such a high level as to require alarge amount of text or images to convey. Typically, a unit would be 2-3paragraphs of information or equivalent description if in any otherform. Of course, in any particular case, a unit may be much smaller ormuch larger.

The processing performed by pre-conversion module I is described withreference to FIG. 1.

Begin

1) Click on icon 11 to import document (text and images). The importeddocument is created using a word processing or equivalent programaccording to well known techniques.

2a) Select a distinct “unit” (example: ear or eye, ) using icon 12 andclick on icon 13 to cut the selected unit carrying the images with it.When a unit is cut, then proceed to step 3 and the selected unit will bepasted into a new file.

2b) Optionally use icon 15, selecting from one of a number of distinctcolors in the palette to color each selected unit to help visualize thenumber of units in a long document. Although six colors are shown inicon 15, by pressing a predetermined key (e.g., shift) on the keyboardof the workstation being used, an additional set of icons with differentcolors may be presented. Then, icon 13 is used to cut the first unit.

3) Click on icon 14 to create a new file for the unit. When a new fileis created, the selected unit from 2a or 2b is pasted onto it.

4) Save the unit in the file by clicking on icon 18. If a document islong and there are, for example, 12 units, then there would be 12 saves.

5) Repeat steps 2-4 until desired portions of the document are brokendown into individual units and saved.

6) After the entire document has been processed as described above, openthe saved “unit” by clicking on icon 16.

7a) If the unit includes only one type of information, use icon 12 toselect the entire document and then click on one of the elements in icon17 to indicate the type of information, i.e., structural (S), time based(T), event based (E), procedural (P) or functional (F). Next, save eachunit for subsequent processing by the appropriate software module S, T,E, P, or F by clicking on icon 18.

7b) If the unit includes two or more types of information, then it isnecessary to select each portion of the document that corresponds toonly one type of information. Next, cut and paste each sub-unit in itsown file. For example, an ear document may contain all categories ofinformation (S T E P or F). Each portion which is assigned a category,is referred to as a sub-unit. Each of the sub-units would then need tobe cut and pasted into their own files and sent to the appropriatesoftware module. It should be noted that these sub-units dealing withthe same unit, in this case the ear, would be saved under a generalfolder “the ear” and each sub-unit will have a file name assigned to it.For example, anatomy of the ear file and physiology of the ear filebelong to the folder “the ear”. Then, save each sub-unit or unit forsubsequent processing by the appropriate software module-S, T, E, P, orF by clicking on icon 18. From here on, sub-units or unit will bereferred to as unit.

End

Of course, the specific icons and arrangements shown in FIG. 1 are notimportant. Further, creating the necessary code to perform each of thesteps described above is well within the abilities of a person skilledin the relevant art. For example, a word processing programming whichoperates in a graphical user interface environment can perform thevarious selecting, cutting and saving operations.

Pre-conversion Module II:

Pre-conversion module II operates to:

1) separate text from images from among the units selected bypre-conversion module I.

2) save the “units” to open up later in one of the S T E P F modules orin the image editor module described below.

The processing performed by pre-conversion module II is described withreference to FIG. 2.

Begin

1) Click on icon 21 to open a “unit” saved in pre-conversion module I.

2) Click on icon 22 to create a new window adjacent to the documentcontaining text and images.

3) Click on icon 23 to drag and drop the images from a text and imagewindow to an image only window. This will leave one window with text andthe other with images.

4) Click on icon 24 to save the new document. The contents of eachwindow will be opened later by the appropriate software module (S T E PF or image editor) for further processing.

End

Of course, the specific icons and arrangements shown in FIG. 2 are notimportant. Further, creating the necessary code to perform each of thesteps described above is well within the abilities of a person skilledin the relevant art.

Module 1:

Module 1 operates to convert structural information into the inventedpictorial/graphical format.

The processing performed by module 1 is described with reference toFIGS. 3, 3 a and 3 b.

Begin

1) Click on icon 31 to open the “structural unit”. This causes the textportion of the unit to be available for further processing.

2) Tagging the heading: Locate the most general text which forms aheading and assign a color to it using icon 32. The choice of color isnot important. Right clicking on the region containing icon 32 providesmore colors. As used herein, selecting or locating text is performedusing a mouse or other pointing device to select text or the like withina document according to well known techniques.

3) Tagging the sub-headings: Locate the text and assign colors to thefewest number of sub-headings possible by using icon 33 color palette.Right clicking on the region containing icon 33 provides more colors.

4) Tagging information related to sub-headings: Select the informationrelated to each sub-heading and click on icon 34 to assign a color toit. Use the same colors from icon palettes 33 and 34 for informationrelated to sub-headings as the color of sub-headings.

5) Tagging transitions: Select transitions using icon 35 and click onone of the elements in icon 38 which corresponds to the location of thetransition. For example, clicking on element 1 of icon 38, places thetransition between the heading and the first sub-heading. Similarly,clicking on element 4 of icon 38, places the transition between theheading and the fourth sub-heading. Right-clicking on icon 38 will causemore numbers (5, 6, 7, 8) to appear and options for colors fortransitions. Since units are small, the number of transition do not tendto go beyond 4.

As an illustration of steps 2-5, reference is made to the followingtext.

A man has a head, trunk, and extremities. The head has two eyes, mouth,two ears, nose and hair. Inside the mouth there are white, pegged shapedobjects called teeth. The mouth also contains a tongue which is fortaste. The ears are for hearing and the nose is for smelling. The trunkhas two parts, an upper trunk and a lower trunk. The lower trunk isusually larger than the upper trunk. The extremities are long and can behairy. The extremities can get cold.

For this text, at step 2, the word “man” is the most general text andwould be selected to form a heading. At step 3, the words “head,”“trunk” and “extremities” would be selected to form three sub-headings.Each would be assigned a different color from icon palette 33. At step4, the words “The head has two eyes, mouth, two ears, nose and hair.Inside the mouth there are white, pegged shaped objects called teeth.The mouth also contains a tongue which is for taste. The ears are forhearing and the nose is for smelling” would be selected as a sub-headingunder “head.” The words “The trunk has two parts, an upper trunk and alower trunk. The lower trunk is usually larger than the upper trunk”would be selected as a sub-heading under “trunk.” Similarly, the words“The extremities are long and can be hairy. The extremities can getcold” would be selected as a sub-heading under “extremities.” At step 5,the word “has” after “man” would be selected as a transition and element1 of icon 38 would be selected to place the transition between theheading and the first sub-heading.

6) Chart application: Create a top-down hierarchical chart by clickingon icon 39. The information tagged in the previous steps will positionthemselves as shown in FIG. 3a.

The heading will branch into sub-headings, the transition will positionitself next to the line connecting heading and sub-headings, theinformation related to each sub-heading will be placed in a boxconnected by a line to the respective sub-headings.

7) Repeat steps 3-5 for each of the sub-headings:

A) If “information related to sub-headings I” is branchable, then

i) Create sub-heading II by repeating step 3.

ii) Create “information related to sub-heading II” (by repeating step 4)

iii) Go to step 8A.

Information is branchable if it can be broken down into furthersub-headings. For example, in FIG. 3a, information in boxes 41 and 42 isbranchable and information in box 43 is not branchable.

B) If information related to sub-heading I is non-branchable, then go tostep 8B.

8) Chart application

A) Select sub-heading II by using icon 35 and then clicking icon 39 ifthe number of sub-headings is three or more, or by clicking on icon 40.

B) Select sub-heading II by using icon 35 and then clicking on icon 40.Each sentence is automatically placed next to an arrow line.

9) Repeat steps 2-8A continuously until information becomesnon-branchable. Then apply step 8B.

FIG. 3b illustrates the results of module 1, steps 1-9.

10) Click on icon 45 (of FIG. 3). This will switch the colors of thefonts to background colors and switch the colors of text to black orwhite to make the chart easier to read.

11) Balancing colors: If two background colors are the same but theirinformation is not directly tied together, then either element can beselected using icon 35 and changing its color by clicking on colorpalette of icon 47.

12) Save the document by clicking on icon 49.

End

Of course, the specific icons and arrangements shown in FIGS. 3, 3 a and3 b are not important. Further, creating the necessary code to performeach of the steps described above is well within the abilities of aperson skilled in the relevant art.

Module 2:

Module 2 operates to convert functional information into the inventedpictorial format.

The processing performed by module 2 is described with reference toFIGS. 4 and 4a.

Begin

1) Perform the processing of module 1 except as follows.

2) Click on icon 51 to automatically attach numbers next to eachsub-heading and to make a line between each sub-heading. The numbers andarrows linking the sub-headings provide information about the flow ofinformation and the number of steps involved.

3) Click on icon 49 to save this document.

End

FIG. 4a shows the results of the operation of module 2 on the followingtext.

The nephron has 6 parts: Proximal convoluted tubule. The PCT absorbs100% of the goodies & 60% of NaCl & water. The loop of Henle absorbs 20%of salt and water. NaCl moves into the descending limb. Ascending limbis where water cannot be reabsorbed. Distal convoluted tubule is the 5thpart. The collecting duct is not permeable to water. However, it becomespermeable in the presence of ADH.

Of course, the specific icons and arrangements shown in FIGS. 4 and 4aare not important. Further, creating the necessary code to perform eachof the steps described above is well within the abilities of a personskilled in the relevant art.

Module 3A:

Module 3A operates to convert procedural information into the inventedpictorial/graphical format.

The processing performed by module 3 a is described with reference toFIGS. 5a-5 g.

Begin

1) Utilize the same steps as described above with respect to structuralinformation until the step of applying the chart to sub-heading, i.e.,icon 39 of FIG. 4.

As an illustration of steps 2-6, reference is made to the followingtext.

There are 4 steps in graphing a straight line. How many points needed tomake a line? Answer 2 points. So, we're going to plot 2 points and thenconnect them. Here are the steps. First, make sure the equation is inthe form y = mx + b. If it isn't make it so by following the bla blasteps. Second, plot on the y-axis. Third, using the slope, plot theother points by following these steps: a) write the slope as a fractionif it already isn't. b) use the following guide: if top is + go up, if -go down, (starting at b) & if bottom is + go right, if - go left.Fourth, join the two points together.

2) When applying the chart to a sub-heading by clicking on icon 51 inFIG. 5a, the chart in this case is designed to grow vertically. The leftover “untagged information” 57 is kept on the side as shown in FIG. 5b.

3) Dealing with the “untagged information”: Next, if the “untagged”information provides some type of orientation before a particular step,use one of the elements of icon 53 to select and give the location ofthe orientation to be attached to the pictorial representation as shownin FIG. 5c.

Element 1 of icon 53 will position the “orientation” between the headingand sub-heading 1.

Element 2 of icon 53 will position the “orientation” between sub-heading1 and sub-heading 2 and so on.

4) Applying Chart to information related to sub-headings:

Next, by holding down on the shift key and clicking on any of the steps(in this case steps 1, 2, and 4), and then clicking on icon 55 to getthe results shown in FIG. 5d.

5) Continue applying chart to information related to sub-headings:Continue applying chart to sub-heading information until all sub-headinginformation has been charted by selecting icon 55 to get the resultsshown in FIG. 5e.

6) Attachment of children to parent:

Next, if the sub-heading information can be organized into a“parent-child” relationship, then follow these steps:

a) Highlight the children and click on icon 59, element 1. This will tagthe children.

b) Next, highlight the parent and click on icon 59, element 2. This willtag the parent.

c) Next, to connect the parent with the children, click on icon 59element 59 a to produce the results shown in FIG. 5f.

d) Next, if there is further description of any of the parts is desired,select the description and click on icon 61 to get the results shown inFIG. 5g. If it is not desired to illustrate the four level descriptionusing branching arrows, just select icon 61.

End

Of course, the specific icons and arrangements shown in FIGS. 5a-5 g arenot important. Further, creating the necessary code to perform each ofthe steps described above is well within the abilities of a personskilled in the relevant art.

Module 3B:

Module 3B operates to convert procedural information into the inventedpictorial format as in module 3A, but module 3B is particularly adaptedfor conveying directions or geographical locations as a special case ofprocedural information into the invented pictorial format:

The processing performed by module 3B is described with reference toFIGS. 5h-5 k and with reference to the following text:

Directions to UCLA. Take Pico going East, then Left on Bundy, then righton Santa Monica, then left at Westwood, when you get to the info booth,pay $5 for parking & they'll give you direction on parking.

Begin

1) Open document to be converted using icon 31.

2) Tag/highlight heading using icon 32, i.e., the words “Directions toUCLA”.

3) Highlight every direction using icon 33 color palette, assigning adifferent color to each direction. In this case there are fivedirections as follows: 1) “Take Pico going East”, 2) “Left on Bundy”, 3)“right on Santa Monica”, 4) “left at Westwood”, 5) “when you get to theinfo booth, pay $5 for parking & they'll give you direction on parking”.

4) Apply chart template by clicking on icon 61. The numbers and boxesand lines will automatically be created as shown in FIG. 5i. Selectuntagged information using icon 35 and delete the untagged informationif it is not part of the direction, i.e., in this case, the severalinstances of the word “then.”

5) Click on icon 63 to change background color to color of text and thecolor of text to black.

6) Optionally place transitions (miles, time, numbers of light, streetsigns, . . . ) using icon 65 to produce results as shown in FIG. 5j.

7) Click on icon 67 to map the steps to a schematic pictorialrepresentation of a road map as shown in FIG. 5k.

Features of the schematic road maps: according to the invention are thatthey:

a) are schematic

b) only use street names

c) arrows to show direction of flow.

End

Of course, the specific icons and arrangements shown in FIGS. 5h-5 k arenot important. Further, creating the necessary code to perform each ofthe steps described above is well within the abilities of a personskilled in the relevant art.

Module 4: Conversion of time based information into pictorial format

Module 4 operates to convert time based information into the inventedpictorial format.

The processing performed by module 4 is described with reference toFIGS. 6-6d and with reference to the following text:

The Ninja War In 1980, The Ninja Turtles took over the south part ofJamaica. The head Ninja was directed by Sergeant Nini who was big,strong, and his neck was the longest. He carried a Ninja weapon built inJapan. The Jamaicans fought the battle for a good 5 years. There were9,000 Ninja casualties with over 35 Jamaicans. In 1985, Sergeant Niniordered his troops to go on top of trees where they would have a betterangle of their enemy. In 1987, the Birdies joined the Jamaicans againstthe invasion of Ninja Turtles. The new weapons used by Jamaicans wasBird droppings. Bird droppings finally drove the Ninja Turtles out byearly 1990s.

Begin

1) Same as steps 1-10 of module 1. The only difference is that at step3, the dates or times corresponding to significant time events arehighlighted by selecting them using icon 35 and then assigning them assub-headings by using icon color palette 33. Then informationcorresponding to the significant time events is selected and assignedcorresponding colors from icon color palette 34 as in step 4 of module1. The result will be as shown in FIG. 6a. Then, using icon 35, selectthe information corresponding to sub-headings and click on icon 40. Thiswill convert the pictorial representation shown in FIG. 6a where eachsentence becomes separated as illustrated in FIG. 6b.

2) Each sentence can be further broken down by highlighting a parentusing element 1 of icon 71 and highlighting a child using element 2 oficon 71. Then click on icon 71 a to map the relation. The results areshown in FIG. 6c

3) Click on icon 73 to place arrows indicating direction of timeline asshown in FIG. 6d.

4) Click on icon 45. This will switch the colors of the fonts tobackground colors and switch the colors of text to black or white tomake the chart easier to read.

5) Save the document by clicking on icon 49.

End

Of course, the specific icons and arrangements shown in FIGS. 6-6d arenot important. Further, creating the necessary code to perform each ofthe steps described above is well within the abilities of a personskilled in the relevant art.

Module 5:

Conversion of event information into pictorial format

Module 5 operates to convert event based information into the inventedpictorial format.

The processing performed by module 5 is described with reference toFIGS. 7-7e and with reference to the following text:

The Ninja War The beginning of the battle occurred in summer of 1980.The Ninja Turtles took over the south part of Jamaica. The head Ninjawas directed by Sergeant Nini who was big, strong, and his neck was thelongest. He carried a Ninja weapon built in Japan. The Jamaicans foughtthe battle for a good 5 years. There were 9,000 Ninja casualties withover 35 Jamaicans. The middle of the battle occurred some time in 1985,Sergeant Nini ordered his troops to go on top of trees where they wouldhave a better angle of their enemy. The stage before the end of the warwas marked by 1987, the Birdies joined the Jamaicans against theinvasion of Ninja Turtles. The new weapons used by Jamaicans was Birddroppings. The end of the battle occurred when Bird droppings finallydrove the Ninja Turtles out by early 1900s.

Event based information has two types of possibilities:

1) structural events

2) flowing events

Structural Events

The steps employed to create pictorial representations for structuralevents is the same as steps 1)-4) of the time based events according tomodule 4 except that instead of selecting the dates or timescorresponding to significant time events and highlighting and assigningcolors to them as described above, and then associating thecorresponding information, the major events or turning points areselected as the subheadings, and the corresponding information isassociated with the subheadings as described above. The result is shownin FIG. 7a.

Flowing Events

Begin

1) Start with structural event already developed as described above andshown in FIG. 7a.

2) Select the section under sub-heading 1 using icon 75 thereby creatingselection box 79 as shown in FIG. 7b.

3) Click on icon 77 to automatically change to “flowing event” format asshown in FIG. 7c which shows the result for only the information inselection box 79. This may be repeated for the information pertaining tothe other significant events corresponding to the other sub-headings.

4) Click on icon 45 to add color background to the text.

End

If the event takes two or more paths, use event bifurcation tool icon81.

Begin

1) Highlight the starting point of bifurcation using labeled 1 of icon81.

2) Select and highlight the start and finish of each of the first pathsof the flowing events using element 2 of icon 81 to produce the resultsshown in FIG. 7d.

3) Select and highlight the start and finish of the second path of theflowing events using element 2 of icon 81.

4) If there are more paths, continue highlighting the start and finishof each part using element 3 or a higher numbered element of icon 81.Right-clicking on the region containing icon 81 provides more numbersand colors.

5) Click on the chart element portion of icon 81 to produce the resultsshown in FIG. 7e.

6) Save the document by clicking on icon 49.

End

Of course, the specific icons and arrangements shown in FIGS. 7-7e arenot important. Further, creating the necessary code to perform each ofthe steps described above is well within the abilities of a personskilled in the relevant art.

Module 6: Conversion of Images

Module 6 operates to convert images into the invented pictorial format.

The processing performed by module 6 is described with reference toFIGS. 8-8j. There are two possible formats. Both formats may bepresented giving the user the option to view both formats.

Format 1 is used where it is desired to display images in a hierarchicalarrangement.

Begin

1) Import text unit by clicking on icon 81 of FIG. 8a.

2) Create or obtain picture applicable to heading=“heading picture”.

3) Create or obtain pictures applicable to sub-headings=“sub-headingpictures”.

4) Create or obtain pictures applicable to sentences=optional dependingon availability of pictures.

Steps 2), 3) and 4) are needed only if it is desired to utilize imagesin addition to any obtained by step 1.

5) Next, import the images by clicking on icon 83 (FIG. 8a). The resultof steps 1-5 is shown in FIG. 8b where portion 85 is the unit savedafter module 1 processing and imported by clicking on icon 81 andportion 87 is the images obtained during pre-conversion module II.

6) Use icon 91 (FIG. 8a) to link the images to their corresponding textelements as shown in FIG. 8c by connecting the text and image elementsusing a mouse or other pointing device.

7) Click on icon 93 (FIG. 8a) to produce a chart from the images asshown in FIG. 8d.

End

Format 2 is used where it is desired to display images in an explodedview format.

Begin

Steps 1-5 are the same as described above with reference to FIGS. 8a-8 dand Format 1.

6) Referring to FIGS. 8e and 8 f, color code information on textidentifying images (i.e., the words “head”, “trunk”, “extremities”,etc.), corresponding to text information (i.e., description shown intext boxes) by using icon 101 color palette.

7) Use either element in icon 103 to create a parent-child relationshipamong images. The only difference between the elements in icon 103 is touse dotted lines if lines cross as shown in FIG. 8g to make it easier tofollow the images connected by the lines.

8) Click on icon 105 to apply chart to images as shown in FIG. 8h andsave by clicking on icon 49. This is done to make the parent-childrelationship easier to follow.

9) Color code image parts to its corresponding text unit as shown inFIG. 8i using icon 107 and save using icon 49. This is to color theimages with the same color as the text describing the image parts.

10) Create fewer colors using icon 109: a) Color the information relatedto sub-headings level 1 equal to the color of sub-headings 1. b)Color-code the images corresponding to colors of the text unit as shownin FIG. 8j and save using icon 49. This is to show the end user how theimages and text relate to each other at a high level.

End

Of course, the specific icons and arrangement of the icons shown inFIGS. 8a-8 j are not important. Further, creating the necessary code toperform each of the steps described above is well within the abilitiesof a person skilled in the relevant art.

Module 7: Linking the invented pictorial/graphical text and imagestogether, if desired.

Module 7 operates to link text converted by one of modules 1-5 andimages converted by module 6 to form a single text with images documentin the invented pictorial/graphical format.

The processing performed by module 7 is described with reference toFIGS. 9-9g.

For all three forms of images saved by module 6 (i.e., FIGS. 8h, 8 i and8 j), perform the following steps where the three forms are three textand image formats referred to as format 1, format 2 and format 3. Informat 1, as shown in FIG. 9c, the images are arranged hierarchicallywith associated text. In format 2, as shown in FIG. 9e, the images arearranged in an exploded view with associated text preserved with lowlevel associations. In format 3, as shown in FIG. 9g, the images arearranged in an exploded view with associated text preserved with highlevel associations.

The three different formats are used to provide alternate mechanisms forviewing the same information enhancing the meaning of the content of thedocument by showing how the parts of information relate to each other indifferent orientations.

Begin

1) Import the pictorial/graphical text and image of type format 1 byclicking on icon 111 as shown in FIG. 9. This will produce results asshown in FIG. 9a.

2) Click on icon 113 to link text to images to produce the results shownin FIG. 9b. The links are formed by manually connecting the desired textand images using a mouse or other pointing device.

3) Click on icon 115 to link text to images to automatically produce theresults shown in FIG. 9c.

End

For format 2, the steps are the same, but step 2 produces the resultsshown in FIG. 9d and 9 e.

For format 3, the steps are the same, but step 3 produces the resultsshown in FIG. 9f and 9 g.

Of course, the specific icons and arrangements of icons shown in FIGS.9-9g are not important. Further, creating the necessary code to performeach of the steps described above is well within the abilities of aperson skilled in the relevant art.

Module 8: Collaboration—creating multiple hierarchies

Once processing by modules 1-6 has been completed, module 8 operates tobring together separately created information blocks into a singleinformation block. To do this, the first step is to collapse eachseparate information block so that each is represented by a single icon.Techniques to do this are well known to persons skilled in the art.

Begin

1) Click on icon 121 to import the “collapsed units” needed to be linkedand turned into a hierarchy. Assuming it is desired to create aninformation block relating to the five senses, the result is shown inFIG. 10a.

2) Click on icon 123 to import the “framework for the units”. This wascreated by one of the S,T,E,P, or F conversion modules. The result isshown in FIG. 10b.

3) Click on icon 81 to highlight the “collapsed unit,” assigning adifferent color to each of the units shown in FIG. 10c.

4) Click on icon 83 to highlight the “framework for the units” so thatthe information blocks 141 are associated with a corresponding frameworkunit 143 as shown in FIG. 10d.

5) Click on icon 91 to link the “collapsed units” to their respectiveposition in the “framework” by using a mouse or other pointing device asshown in FIG. 10e.

6) Click on icon 93 to integrate the “collapsed units” into theframework as shown in FIG. 10f.

7) This entire framework can itself be collapsed and integrated into ayet larger framework, which repeats the process starting with Step 1.For example, the five senses can be collapsed and integrated into alarger framework called Sensory and Motor Systems, which in turn couldbe collapsed and integrated into a larger framework or module calledAnatomy. Anatomy could further be collapsed and integrated into a largermodule called Biological Sciences. Biological Sciences can be collapsedand integrated with other “mega-modules” such as Physical Sciences andEarth Sciences to create Science. The Science module can be furthercollapsed with other supersets of modules such as Math to create Mathand Sciences. Math and Sciences modules may further be collapsed andintegrated with History and Humanities as well as other modules tocreate World Knowledge. This modular process can be repeated multipletimes, thus allowing for massive scalability. This also represents a“bottom-up” development of content.

The structure of the framework may be modified if desired. For example,if the number of sub-headings is five, the author may use the tools 133to create a smaller number of sub-headings by creating hierarchies asshown in FIG. 10g. That is, clicking on one of the tools forming icon133 initiates an editing module which is described below.

End

Of course, the specific icons and arrangements shown in FIGS. 10-10g arenot important. Further, creating the necessary code to perform each ofthe steps described above is well within the abilities of a personskilled in the relevant art. Also, although the example uses thestructural category example, the same processing would apply to thetime, event, procedural and functional categories as well.

Module 9: Editing Layout, Content and/or Format

In the preferred embodiment, after the processing described above hasbeen completed, to provide additional information in the views whichwere created, or to clean-up or otherwise enhance the output which hasbeen generated, the created files may be edited. Specifically, thelayout may be edited, the content may be edited and/or the format may beedited.

To edit the layout, the prioritization, hierarchies within units and/orthe structure may be edited. To edit the prioritization, where a numberof blocks of information appear in a vertical arrangement such as shownin the first set of information blocks on the left side of FIG. 6b, anumber (e.g., from 1-4) can be assigned to each block of informationwhich will result in a reordering of the vertical arrangement of theblocks based upon the numbers assigned. Alternatively, a block ofinformation can be selected and then dragged and dropped to a newposition. To edit hierarchies within units, again, assuming a verticalarrangement of blocks as in the first set of information blocks on theleft side of FIG. 6b in which all blocks are within a single hierarchy,first select how many hierarchies are desired (e.g., from 2-7). If forexample two hierarchies are desired, then two branches would be created,each with a blank space for a title. All the blocks of information wouldappear under the first branch. Then select the block or blocks whosehierarchy it is desired to change and cut and paste or drag and dropthese selected block or blocks to the second branch. Then type anappropriate title for the two branches.

Hierarchies may also be added where the information blocks arehorizontally arranged as in FIG. 6b, taking all of the informationblocks from left to right into account. To edit the structure, existinginformation blocks are broken up to form a parent child relationship.For example, in FIG. 6b, one information block reads as follows:

The head Ninja was directed by Sergeant Nini who was big, strong, andhis neck was the longest and a second information block reads asfollows: He carried a Ninja weapon built in Japan

Assume it is desired to form a parent child relationship with the words“Sergeant Nini” being the parent and the words “was big, strong, and hisneck was the longest” being one child and the words “He carried a Ninjaweapon built in Japan” being a second child. To create thisrelationship, the words “Sergeant Nini” would first be selected andassigned as the parent, by for example, clicking on a P icon. Then thetwo children would be selected and assigned as the children by, forexample, clicking on a C icon. Then an icon such as 59 a shown in FIG.5a would be selected to form the parent with two children hierarchy.

To implement the various editing techniques described above or othertechniques to change prioritization, hierarchy or structure created bythe modules as discussed above would be well within the abilities of aperson skilled in the relevant art.

The content, i.e., language, transitions, orientation, patterns andbalance may also be edited. The language could be edited using wellknown word processing techniques. Transitions, orientation, patterns andbalance may be edited using techniques used in a variety of graphicsoriented programs.

The format, i.e., use of symbols instead of text (e.g., “+” instead of“plus” or 35° instead of thirty-five degrees) can be edited using wellknown word processing techniques.

Module 10: Knowledge Level Conversion

Currently, problems exist in navigation and information acquisition withrespect to learning more details with respect to a set of information.

1. One way this is handled is to select a level based on beginning,intermediate and advanced levels. However, from the end-user'sperspective, having to select a level based on beginning, intermediate,advanced levels is not the ideal way of accessing the information. Thereare several problems with this approach:

a) when viewing the information as pages on a monitor, there isinterruption and the necessity to jump to a completely different page;

b) this approach does not readily recognize what was new and what theuser already covered in previous level.

2. From a development/business perspective:

The cost of conversion increases because of having to author fourversions of the same information and saving them in four differentlevels.

In general, an end-user does not want to keep changing selectionsbetween beginner, intermediate, advanced levels and keep changing backand forth as the user wants more or less detail. It is preferable from auser standpoint to choose a level (e.g., 1-5) and then simply select thelayout to view at any of the levels or to select the level of detail ofa portion of the layout:

The same author who creates a knowledge level 1 set of information, mayperform the following algorithm to create additional levels:

1. recognize the only difference between topics for different knowledgelevels is the amount of detail within a unit of information andconnecting links. That is, knowledge level conversion should not betreated with distinct separate levels with a clear cut break betweengroups of users—rather, knowledge level conversion is nothing more thana more detailed explanation/map and their connecting units ofinformation or layouts.

2. recognize each unit of information or layout will be tagged with fivelevels of detail.

3. recognize each topic accessed by the user will display a unit ofinformation or layout based on the level of detail chosen.

4. recognize each new unit of information or layout that the userrequests will have its own accessing the information and at any layoutcan view the information in any of the five knowledge levels ofdifficulties.

As an example, assume that a unit of information is assigned level 3. Amore detailed view of that unit will begin with a new numbering of eachinformation unit within the more detailed view from level 1 to level 5.It should be noted that knowledge conversion may be performed regardlessof whether the original document is all text, all images or acombination of text and images.

Once processing by modules 1-6 has been completed, module 10 may be usedto perform knowledge level conversion as outlined above as describedbelow with reference to FIGS. 12-12e.

Begin

1) Click on icon 31 to open the unit for which it is desired to performknowledge level conversion. The result is shown in FIG. 12a.

2) Click on icon 34 and use the pointer to select the nodes and arrowhead portions for which it is desired to assign a knowledge level. Theresult is shown in FIG. 12b.

3) Click on icon 141 to produce a window with numbers corresponding tolevels to be used for knowledge conversion as shown in FIG. 12c.

4) Select a number according to a desired level of knowledge to beassigned. The result shown is shown in FIG. 12s assuming level 1 wasselected.

5) Repeat assigning numbers as in step 4 for each heading and child toproduce the results shown in FIG. 12e.

6) Click on icon 49 to save the document.

Once this conversion has been performed, a user can be presented withmore or less detailed information depending selected level. This may beimplemented in a view mode by the user selecting a heading for which itis desired to see more detail and then choosing the level of detaildesired. Of course, the chosen level of detail can be changed by theuser as desired.

End

Of course, the specific icons and arrangements shown in FIGS. 12-12e arenot important. Further, creating the necessary code to perform each ofthe steps described above is well within the abilities of a personskilled in the relevant art.

Further processing, which is optional, may then be performed to addmultimedia components.

While there has been described and illustrated specific embodiments ofthe invention, it will be clear that variations in the detailsspecifically illustrated and described may be made without departingfrom the true spirit and scope of the invention as defined in theappended claims when interpreted in accordance with the full breadth towhich the claims are legally and equitably entitled.

I claim:
 1. A method for converting a document which contains pluralityof text and pictures which is intended to convey information in atraditional, linear format into a new structural document for display inwhich the text and pictures are rearranged so that the information islaid out in a pictorial representation, said method using a computerhaving a monitor or other display device and comprising: accessing apreviously created document such that it is displayed on a monitor orother display device; selecting text portions of the document andassigning one of a set of categories to each selected portion forsubsequent processing, wherein the categories are one of: i)structural,ii) functional, iii) procedural, iv) time based, and v) event based;rearranging and maintaining the selected text portions; processing theselected text portions of the document based upon the assigned categoryso as to arrange the selected text portions so they appear on themonitor or other display device in a pictorial arrangement, wherein theselected text portions can still be read after being arranged in saidpictorial arrangement; selecting portions of the displayed pictorialarrangement and assigning a knowledge level to each of the selectedportions; processing selected pictures within said document so as toarrange the pictures in one of a hierarchical and logical exploded viewformat, and linking the processed selected text portions and pictures.2. The method defined by claim 1 wherein the assigned category is basedon the subject matter of the text content of the document.
 3. The methoddefined by claim 1 wherein said linking comprises: importing theprocessed text and pictures; manually connecting the desired text andpictures using a mouse or other pointing device to identify desiredlinks between the processed text and pictures; processing the manuallyconnected text and pictures to display results of said processing. 4.The method defined by claim 1 wherein said accessing, selecting andprocessing are each initiated by selecting a predetermined icon on saidmonitor or other display device.
 5. The method defined by claim 3further comprising: editing at least one of layout, content and formatof said displayed results of said processing.
 6. An article ofmanufacture comprising: a computer usable medium having computerreadable program code means embodied therein for converting a documentwhich contains a plurality of text and pictures which is intended toconvey information in a traditional, linear format into a new structuraldocument for display in which the text and pictures are rearranged sothat the information is laid out in a pictorial representation, thecomputer readable program code means in said article of manufacturecomprising: computer readable program code means for accessing apreviously created document such that it is displayed on a monitor orother display device; computer readable program code means for selectingtext portions of the document and assigning one of a set of categoriesto each selected portion for subsequent processing, wherein thecategories are one of: i) structural, ii) functional, iii) procedural,iv) time based, and v) event based; computer readable program code meansfor rearranging and maintaining the selected text portions; computerreadable program code means for processing the selected text portions ofthe document based upon the assigned category so as to arrange theselected text portions so they appear on the monitor or other displaydevice in a pictorial arrangement, wherein the selected text portionscan still be read after being arranged in said pictorial arrangement;computer readable program code means for selecting portions of thedisplayed pictorial arrangement and assigning a knowledge level to eachof the selected portions; computer readable program code means forprocessing selected pictures within said document so as to arrange thepictures in one of a hierarchical and logical exploded view format, andcomputer readable program code means for linking the processed selectedtext portions and pictures.
 7. A method for converting a document whichcontains a plurality of text and pictures which is intended to conveyinformation in a traditional, linear format into a new structuraldocument for display in which the text and pictures are rearranged sothat the information is laid out in a pictorial representation, saidmethod using a computer having a monitor or other display device andcomprising: accessing a previously created document such that it isdisplayed on a monitor or other display device; selecting pictureportions of the document and assigning one of a set of categories toeach selected portion for subsequent processing, wherein the categoriesare one of: i) structural, ii) functional, iii) procedural, iv) timebased, and v) event based; selectively linking said selected pictureportions with rearranged text; processing the selected picture portionsof the document based upon the assigned category so as to arrange theselected picture portions and the rearranged text so they appear on themonitor or other display device in a pictorial arrangement, wherein therearranged text can still be read after being arranged in said pictorialarrangement; selecting text portions of the displayed pictorialarrangement and assigning a knowledge level to each of the selected textportions; processing selected pictures within said document so as toarrange the pictures in one of a hierarchical and logical exploded viewformat, and linking the processed selected text and picture portions. 8.The method defined by claim 7 wherein the assigned category is based onthe subject matter of the picture content of the document.
 9. The methoddefined by claim 7 further comprising: creating or obtaining a pictureexternal to the document and assigning one of said categories from saidset to said external picture for subsequent processing.
 10. The methoddefined by claim 7 wherein said accessing, selecting and processing areeach initiated by selecting a predetermined icon on said monitor orother display device.
 11. The method defined by claim 7 furthercomprising: editing at least one of layout, content and format of saidpictorial arrangement.
 12. An article of manufacture comprising: acomputer usable medium having computer readable program code meansembodied therein for converting a document which contains a plurality oftext and pictures which is intended to convey information in atraditional, linear format into a new structural document for display inwhich the text and pictures are rearranged so that the information islaid out in a pictorial representation, the computer readable programcode means in said article of manufacture comprising: computer readableprogram code means for accessing a previously created document such thatit is displayed on a monitor or other display device; computer readableprogram code means for selecting picture portions of the document andassigning one of a set of categories to each selected portion forsubsequent processing, wherein the categories are one of:i) structural,ii) functional, iii) procedural, iv) time based, and v) event based;computer readable program code means for selectively linking saidselected picture portions with rearranged text; computer readableprogram code means for processing the selected picture portions of thedocument based upon the assigned category so as to arrange the selectedportions and the rearranged text so they appear on the monitor or otherdisplay device in a pictorial arrangement, wherein the rearranged textcan still be read after being arranged in said pictorial arrangement;computer readable program code means for selecting text portions of thedisplayed pictorial arrangement and assigning a knowledge level to eachof the selected text portions; computer readable program code means forprocessing selected text pictures within said document so as to arrangethe pictures in one of a hierarchical and logical exploded view format,and computer readable program code means for linking the processedselected text and picture portions.